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PREFACE 


This booklet contains a variety of comments regarding my 
tent Theory of Computation ; Formal Languages . Automata , and 
Complexity along with some ideas and transparency masters that a 
teacher of a course based on that text may want to use. The 
booklet contains a chapter corresponding to each chapter in the 
parent text and concludes with a section of transparency masters. 
Within each chapter are sections entitled General Comments, 
Comments Regarding Chapter Review Problems, and Additional 
Problems. 

The General Comments sections contain comments regarding 
such topics as why things are presented in the text as they are, 
what sections can be skipped if time is limited, suggested 
project subjects, points on which students tend to stumble, and 
some approaches that have worked for me. The section called 
Comments Regarding Chapter Review Problems is a collection of 
comments about the problems in the text. This is not an answer 
key for grading homework. Instead, its purpose is to help an 
instructor decide which problems should be assigned. For 
example, it indicates which problems are designed to extend the 
material in the text, identifies wrong answers that I have seen 
and the misconceptions on which they were based, discusses 
various approaches to the problems, and at times provides 
answers. 

I have always questioned the validity of test banks, 
especially those full of multiple-guess and trick-or-false 
questions. An exam should be custom made to reflect the course 
environment and the aptitude of the students. On the other hand, 
an instructor’s creative energies can be drained at test 
preparation time. Thus, the Additional Problems section in each 
chapter of this guide contains ideas for problems. My goal here 
is to give some examples of questions that are different from 


/ 









those in the text. Not all of them are necessarily good test 
problems—some are too time consuming and others produce answers 
that are not easily classified as right or wrong. However, all 
of these problems could serve as catalysts at test preparation 
time. 


The transparency masters found at the end of this booklet 
are selected from the figures in the parent text. I hope they 
are useful. 


J.0.8. 
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Chapter © 


PRELIMINARIES 


General Comments 


1. My first comment regarding this chapter relates to the 
entire text. My goal in writing this book was to give students 
an understanding of the scope of the subject—not to make them 
experts on the details. However, your students will probably 
spend the entire semester wanting to slow down. They will think 
they have to master a topic before moving on. If you give in to 
this pressure, your students will still be drawing transition 
diagrams for finite automata at the end of the semester and will 
not have experienced the excitement of seeing how this material 
relates to other fields. They will leave the course convinced 
more than ever that theory is a bunch of abstract nonesense. I 
urge you to keep moving fast enough so that your students will 
recognize the plot rather than merely becoming familiar with some 
of the characters. 

2. I assume that students reading this text have already taken 
courses in discrete mathematics and structured programming. 
Thus, the purpose of this introductory chapter is merely to set 
the stage for the study to follow--not to present material that 
is new to the students. If your students have never seen the 
rudiments of set theory or need to learn the technique of proof 
by induction, you will probably need to supplement the material 
in this chapter land will probably not finish the text in one 
semester). 

A likely exception is the material regarding cardinality of 
sets and the distinction between countable and uncountable sets. 
I suspect that this material will be new (and difficult) for most 
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students. Do not, however, allow this to slow you down. As I 
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said before, the material in this chapter should be considered 
preliminary to the course--not a part of it. Instead of dwelling 
on the issue of countable versus uncountable until all your 
students feel secure with the idea, merely present the 
distinction once, along with the observation that there must be 
more problems to solve than there are programs that can be 
written. Then, move on to Chapter 1. (I usually hit the high 
points of Chapter 0 on the first day of class and start Chapter 1 
on the next class day.) The distinction between countable and 
uncountable sets will surface several times throughout the 
course, and each time more students will catch on. Spending more 
time at the beginning of the semester will make little difference 
in their understanding and will cause other, more important, 
topics to be omitted. 

Basically, a student should be prepared to understand the 
following hierarchy: 

fin alphabet E is finite. 

The collection E of all strings of finite length is 

countably infinite. 

ft 

The collection of all languages over E (all subsets of E > 
is uncountable. 

3. The presence of such terms as Schroder-Bernstein Theorem and 
Axiom of Choice makes this chapter appear more technical than it' 
needs to be. fill that is needed from this chapter is an 

intuitive understanding of the ideas presented. The reason. I 
resorted to quoting such items as the Axiom of Choice was that I 
felt the need to establish a mature dialogue from the beginning. 
Students at this level easily slip into a very informal (and 
incorrect) “proof by example" mode. Thus, I felt it better to 
start from a solid, somewhat formal chapter and td loosen the 
approach later than to allow too much informality in the 
beginning and then have to tighten the ropes as more rigor was 
required. 
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4. The text introduces the concept of a computational hierarchy 
through the analogy of a physicist who must expand the physical 
laws of Newton to solve more complex problems. Another example 
that I like to use as a motivating tool (and with which students 
seem to identify) is based on a high school algebra student who 
solves problems by first finding an algebraic formula for solving 
the problem and then applying that formula. For example, to 
change a temperature in Centigrade to the equivalent Fahrenheit 
reading the student vrould apply the formula F = (9/5)C + 32. 
But, are there problems that cannot be solved by this means? 
Yes. For example, there is no general purpose algebraic formula 
for computing the roots of polynomials with degree five. Another 
example is the problem of computing the instantaneous slope of a 
curved line. 

Thus, the tools of the high school algebraist are limited, 
and one might ask how these tools could be expanded to obtain 
techniques with more computational power. (One such extension is 
to add the ability to compute limits, which leads to the subject 
of the calculus.) 

Now consider a computer programmer who solves problems by 
expressing a solution in a particular programming language and 
then running the program obtained. Are there problems that 
cannot be solved using this technique? If so, are there ways of 
extending the programming language to obtain more computational 
power? 

5. Interesting examples to use when explaining proof by 
induction are sometimes hard to find. Here is one that is 
gaining in popularity. 

Prove that any "checker board" with 2 n rows and 2 n 
columns from which one square has been removed can be 
covered by L-shaped tiles, each of which cover three squares 
as shown by the following diagram. ./ 
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The base case for this problem is easy, because if n = 1, 
then the board to be covered is exactly the shape of one tile 
(regardless of which square is removed). 



If we now assume that the problem can be solved for any 
board with 2 n rows and 2 n columns, we can show that the problem 
can also be solved for boards with 2 n+ * rows and 2 n+ * columns by 
first considering the board to be divided ( into four quadrants, 
each consisting of 2 n rows and 2 n columns. The hole in the board 
must appear in one of these quadrants. Thus, by our induction 
hypothesis, that quadrant can be covered by tiles. Note that the 
remaining three quadrants must form a large L shape. (See 
diagram below.) Thus, these quadrants can be covered by the 
tiles by first placing a tile at the bend of the L in such a way 
that one square from each of the three quadrants is covered. 
Then, each of these remaining quadrants consists of 2 n rows and 
2 n columns with one of its squares already covered. In turn, our 
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induction hypothesis tells us that these quadrants can also be 
cowered by tiles. 





6. Although many students will find infinite cardinals a new 
and difficult topic, some will be fascinated by it. I have found 
that this provides a good entry point for an independent study 
project focusing on cardinal and ordinal numbers. In this 
context, my students have enjoyed the book Theory of Sets by E. 
Hamke (Dover, 1950). 


7. Many authors present an abstract introduction to languages 
in their introductory chapter. I have not. There are two 
reasons for this. The first is that I consider the introductory 
chapter to be preliminary in nature. The topics that surface 
there that are new for this course are merely mentioned, not 
studied. A more thorough discussion will appear when the topic 
arises in its natural sequence in the course. The second is 
related to my goal of presenting the material in a manner with 
which students at this level can identify rather thaiji lecturing 
them in abstract theories. I don’t think that a junior or senior 


5 

















majoring in computer science is very excited by a course that 
starts with a week's study of closure properties of abstract 
languages. 

I have found that students understand much of the theory of 
languages intuitively once they understand why such a theory 
exists, whereas a preliminary abstract study of languages is 
usually lost on the majority of the students. Why should we 
insist that our students learn a subject abstractly? Why not let 
the subject surface from a discussion of applied problems? "Good 
mathematics" comes from a desire to solve real problems—not from 
theorems derived from randomly defined theories. 


Comments Regarding Chapter Review Problems 

Problems 7 through 14. These are standard induction problems. 
If your students have trouble with them, you will probably need 
to take more time on the inductive proofs that appear later in 
the text than you would normally do. I would, however, caution 
against spending much time in this chapter on induction alone, 
fts I mentioned before, it is very easy to end up exhausting the 
semester on such preliminary topics at the expense of the subject 
of the course. 

I 

Problem 18. This is a straightforward example of a 
diagonalization argument. Such arguments will appear later in 
the text. 1 suggest you assign this problem as a preview of 
things to come. 

Problem 29. Imagine a column in which all the ‘subsets with 
cardinality one appear in "alphabetical" order, another column of 
subsets of cardinality two, then three, etc. Now count the 
collection of all finite subsets starting with the empty set and 
then traversing the columns as shown in the following diagram. 
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First 

column 


Second 

column 


Third 

column 


Count all the 
finite subsets 
in this order. 




► 






Problem 31. Represent the countably infinite set by the points 
on the traditional x/y-coordinate system whose coordinates are 
integers. Draw two vertical lines—one through the point (-1,0) 
and the other through the point (1,0). How, rotate the region 
between these lines about the point (0,0). For each angle 8 in 
the range 0 < 8 ( 2n, let R^ denote the points with integer 

coordinates lying within the region when rotated through an angle 
of 8. The collection CR„: 0 < 8 ( 2n> satisfies the conditions 

C7 

of the problem. That is, each R^ is infinite and if 8 ^ ^ 8 ^ then 
R„ 0 R„ is finite. 


fldditional Problems 


1. Use proof by induction to show that 

5n(n + 


5 + 

for every n in 


10 + 15 + 


+ 5n = 


1 ) 


./ 


i 


! 
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2 . 


Use proof by induction to show that 
2•4 + 4*6 + 6*8 + ••• + 2n(2n + 2) 


4n(n + l)<n + 2) 
3 


for every n in IN + 

3. Use proof by induction to show that n < 2 n for every n in IN. 

4. Use proof by induction to show that 2 n < nf for every 

integer n greater than 4. 

2 

5. Use proof by induction to show that n <2 for every 

integer n greater than 4. 

6. Show that the union of two countable sets is a countable 

set. 

7. Show that there are uncountably many finite subsets of an 
uncountable set. 

8. Show that the union of countably many countable sets is 

countable. 

9. Show that the set of all subsets of IN is uncountable. 

1®. a. List all the functions from the se^ <x,y> into the set 
fa,b>. 

b. List all the functions from the set (x,y> onto the set 
Ca,b>. 


11. a. Show that there are only countably many functions from 
the set Cx,y> into IN. 

b. How many functions are there from the set fx,y> onto IN? 
Support your answer. 
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Chapter 1 


FINITE AUTOMATA 
AND 

REGULAR LANGUAGES 


General Comments 


1. Many students have trouble with the concept of "the language 
accepted by a machine." They think that if a machine accepts all 
the strings in a language then it accepts that language. It is 
important for this misunderstanding be corrected as soon as 
possible. A language is accepted by a machine if all its strings 
and only those strings are accepted by the machine. In 
particular, the finite automaton 



over the alphabet L = tx,y> accepts all the strings of the form 
x n y n , where n € IN, but it does not accept the language fx n y n : 
n € IN> because it accepts strings outside this language as well. 


2. Another common problem dealing with languages is that 
students get the idea that if a language is infinite it must 
contain infinite strings. Perhaps this is merely the result of 
temporary confusion, but it may pay dividends if you emphasize 
the point that all the strings being considered have finite 
length. (As an example, you could remind your students that 
there are infinitely many positive integers, but ( each such 
integer is represented in base ten by a string of finite length.) 

/ 
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3. Several reviewers have commented on my presentation of the 
pumping lemma for regular languages (both pro and con). I have 
chosen to present a specific version of it in the text (Theorem 
1.2) and leave the more general version as an exercise (Exercise 
3 following Section 1.3). (This exercise is a simple 
generalization of Theorem 1.2.) This is because the general 
pumping lemma is not the statement that is actually needed in the 
text's discussion. Moreover, the approach of presenting a 
special case of the lemma first follows the text’s theme of 
teaching the material rather than merely presenting it in a 
definition-theorem-proof format. That is, I think students 
understand results like the pumping lemma by first seeing its 
application in a particular setting. (For those who do not 
subscribe to this philosophy, I have presented the pumping lemma 
for context-free languages in its general form in Chapter 2.) 

Other problems that reinforce the pumping lemma for regular 
languages include Exercise 4 at the end of Section 1.3 and 
Chapter 1 review problems 4, 10, 22, and 23. 

4. I encourage you to try to get your students to consider what 
theorems and problems in the text say about applied situations. 
For instance, there are a lot of problems of the form "Does the 
set of all strings such that constitute a regular language?"*, 
and without guidance students answer these problems without' 
realizing that they are actually asking the question "Can this 
pattern be represented (defined, characterized) by a transition 
diagram, a regular grammar, or a regular expression?" That is 
“Are the tools of transition diagrams, regular grammars, and 
regular expressions powerful enough to identify such patterns?” 

In this regard, you might discuss the design of a typical 
word processor. One feature of this processor should be the 
ability to search for all occurrences of a pattern designated by 
the user. Ask your students what notational system should be 
used to communicate the target pattern to the system? Uhat 
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limitations does the proposed system impose on the patterns that 
can be communicated? (If someone wanted to tell the word 
processor to search for palindromes, how could the concept of a 
palindrome be communicated to the system?) 

5. You may wish to point out the relationship between chapter 
review problems 19 and 26. In fact, the questions raised by 
comparing these problems might serve to launch an inquisitive 
student into independent study. 

6. Many authors provide for X-transitions when defining 
nondeterministic finite automata. This approach simplifies the 
proof of such statements as "the union of two regular languages 
is regular" but significantly complicates the proof that 
nondeterministic finite automata accept only the languages 
accepted by their deterministic counterparts. Thus, I avoided 
the use of X-transitions in the text but did include the concept 
in chapter review problem 14. As a result, you may want to 
assign this problem so that your students will be exposed to 
X-transitions. 

7. I find that students at this level are not experienced in 

solving problems of the form “Is the language regular? 

Support your answer.” They prefer problems of the form "Show 
that the language is regular.” After all, the former 

requires a higher level of understanding than the latter. You 
will note that many of the problems in this chapter (and 
throughout the text) are posed in the more challenging format. 
If your students are having trouble with the material, you may 
ease their burden somewhat by restating these problems into a 
less demanding format. 

8. There are numerous topics relating to this chapter that are 

suitable for independent study projects. Thesje include 

properties of languages, variations of finite automata such as 
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finite transducers, and perhaps semigroups (for the more 
mathematically inclined). 


Comments Regarding Chapter Review Problems 

Problem 2. Use DeMorgan’s laws. That is, 0 = (L° U L^) 0 . 

Problem 4a. Don’t overlook the special case of jrj = 1 in which 
each string in E is a palindrome. 

Problem 4b. This is essentially problem 4a again. You’ll be 
surprised by how many students will answer 4a correctly and 4b 
incorrectly. 

Problem 6. The advantage of having different characterizations 
of the regular languages is that one can choose to use the 
characterization that fits the problem at hand. In this case it 
is best to work with regular expressions. 

Problem 27. Any language is the union of (perhaps infinitely 
many) finite languages, each of which is regular. 

I 

Problem 32. This problem is for your better students. Don’t 
expect a quick, simple answer. 


Additional Problems 

- - • 

1. Give an example of two languages Lj and that are not 

regular such that Lj U L 2 is regular. 
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2 . 


Give an example of two languages and that are not 

regular such that ° L. is regular. 

3. If L is a language for which L is regular, must L itself be 
regular? Support your answer. 

4. Which of the following languages are not regular? Support 
your answer. 

a. fxy, yx, xxy, yyx> 

b. fx^y* 1 : m,n € WJ 

c. tx^y”: m + n = 25> 

, r r s t 

d. Cx y x s s + t a* r> 

5. Draw a transition diagram for a finite automaton that 
accepts the language consisting of those strings in £x,y> that 
do not contain the pattern xxy. 

6. Find a sequence of regular languages L^ f L , L , ••• such 
that L. CL... for each i € IN and U L. is not regular. 

1 1+1 -CM 1 

1 t frl 

7. List all the strings accepted by the finite automaton based 
on the following transition diagram. 


* X 



y y 


8. Describe the language accepted by the finite automaton based 
on the following transition diagram. 
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9. Draw a transition diagram for a finite automaton that 

r s t 

accepts the language fx y x s r 5 s,t € IH>. 

10. Drav; a transition diagram for a finite automaton that 

accepts the language generated by the following grammar (whose 
start symbol is S). 

S -» xyS 
S -» X 
S -» xN 
H -4 zyx 

11. Is there a regular expression that represents the language 
generated by the grammar (whose start symbol is S) below? If so, 
what is it? If not, why not? 

S -4 xMNx 
M 4 xxM 
M -» X 
N -4 yyN 
N -» X 
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Chapter 2 


PUSHDOWN AUTOMATA 
AND 

CONTEXT-FREE LANGUAGES 


General Comments 


1. You will find that describing pushdown automata with 
transition diagrams is a somewhat novel approach. Other texts 
tend to drop the diagram approach when discussing pushdown 
automata and emphasize grammatical techniques instead. 1 have 
found that the procedural approach of transition diagrams is more 
readily accepted by undergraduate students. After all, their 
background has been one of programming in a procedural 
language--a close cousin to designing transition diagrams. 
Moreover, the use of transition diagrams provides a thread of 
continuity through the study of finite automata, pushdown 
automata, and Turing machines. 

2. If you need another example when discussing the distinction 
between general string acceptance and acceptance with an empty 
stack, I suggest the automaton represented below. It accepts the 
language fx m y n : m, n € IN>, but many of these strings are accepted 
with garbage remaining on the stack. If we require that the 
automaton accept strings with its stack empty, then only the 
strings of the form x n y n would be accepted. 






OF course this diagram can be modified as described in the text 
to accept all the strings in the language Fx m y n : m, n € !N>. The 
result is shown below. 



3. The process of constructing a context-free grammar from a 
pushdown automaton as described in the proof of Theorem 2.3 
introduces rewrite rules that may newer be used. This may bother 
some students. I try to ease their anxiety with the following 
points: 

a. Our task is not to find an efficient grammar but merely 
to show that a grammar exists. 

b. The situation is similar to the proof that any language 
accepted by a nondeterministic finite automaton can be accepted 
by a deterministic finite automaton (Theorem 1.3). Indeed, that 
construction often introduces numerous states that may not be 
reachable from the initial state. 

c. Finally, I hawe found that using the following analogy 
helps. The program statement j 

Z := K + Y 

has the potential of being used in many situations. It could be 
used to compute the sum of two integer values; it could be used 
to compute the sum of two real values; it could be used to 
concatenate two character strings; etc. A compiler of the 
language must be prepared to apply this statement to each of 
these applications (or goals). Of course, in any particular 
program, many of these applications may not appear. But, this 
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does not mean that the statement does not have the potential of 
being used in such applications. 

In a similar manner, a transition of the form (p,x,y; q, z) 
has the potential of being applied as a step in satisfying many 
different goals. Just as a compiler is designed to handle any 
situation, we define the grammar to handle any of these goals. 
The fact that some of these goals may never surface when 
executing a particular pushdown automaton does not mean that our 
grammar is wrong, just as a particular program not using the 
statement Z s= K + Y in all its possible ways does not imply that 
the compiler is wrong for being prepared. 

4. Your students may feel that the topic of parser construction 
is not covered as thoroughly as they would like. In most cases 
this indicates that they are wanting an algorithm for 
constructing parse tables. You can explain that such algorithms 
exist but their presentation would be time consuming and belongs 
in a course on compiler construction. In this course the goal is 
merely to show how theoretical topics such as pushdown automata 
form the foundation of more applied subjects in computer science. 
Take this opportunity to encourage students to follow this course 
with one on compiler construction. For now, the exercises 
involving the construction of parse tables are simple enough to 
be solved by brute force. 

5. If you are running short on time you might consider skipping 
the material on Chomsky normal form within Section 2.2. Although 
a traditional topic, this is not a major prerequisite for the 
remaining text. In fact, the only place Chomsky normal form is 
used later in this text i/s in the proof that every context-free 
language is Turing-decidable. You could patch this proof to 
avoid the need for Chomsky normal form or merely skip the proof 
entirely. 

j 
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6. When discussing Chomsky normal form (or even if you choose 
to skip that topic) you may wish to point out that the proof of 
Theorem 2.3 demonstrates another normal form theorem. Every 
rewrite rule obtained in that proof has one of the following 
forms: 


N -> X 
N -» tP 
N -» tPQ 

Thus, this proof shows that every context-free language can be 
generated by a grammar containing only rewrite rules with these 
formats. 

7. Here is an interesting problem to pose to your students when 
discussing the technicalities of deterministic pushdown automata. 
Ask them to try to develop a deterministic pushdown automaton 
that simulates the program segment 

if (NextSymbol = x) then read the x 

else read nothing and pop a y 

At first glance this looks like a simple task, but in reality it 
relies on the lookahead principle. , 

8. Topics for independent study projects relating to this 
chapter include parsing techniques, the construction of LL and LR 
parse tables, normal forms of grammars, and context-sensitive 
languages. 
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Comments Regarding Chapter Review Problems 


Problem 3. Based on the material covered in the text, a rigorous 
proof of part c of this problem is more difficult than parts a 
and b. 

Problems 8 and 9. These are good problems for making sure that 
string acceptance by pushdown automata is understood. 

Problems 12 through 15. These grammars are simple because the 
idea is for students to construct the parse tables by brute force 
methods. My experience has been that once these problems are 
assigned, the students start asking questions about how parse 
tables for real languages are constructed. Many leave this 
chapter with plans to take a compiler construction course as soon 
as possible. 

Problem 19. This is a nice reminder of Chapter 1. 

Problem 23a. I have found that students first try to solve this 


problem by 

developing a 

pushdown 

automaton. 

(This is 

the 

procedural 

’’programming " 

approach 

with 

which 

most are 

more 

comfortable. 

> You may 

want to 

take 

this 

opportunity 

to 

demonstrate 

the importance 

of looking 

at a 

problem from different 


points of view. More precisely, the simple grammar 

S -* xSy 
S -» xSyy 
S -* X 

does the job in this case. 

Problem 28. Every language is the union of (perhaps infinitely 
many) finite languages. Thus, the answer is no. But, the union 
of a finite number of context-free languages is context-free. 

i 
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Problem 35. This is the only reference to Griebach normal form 
in the teHt. If you want to expose your students to this topic, 
now is your chance. 


Additional Problems 


1. Summarize the effects of requiring determinism and the empty 
stack condition on language acceptance by pushdown automata. 

2. Design a context-free grammar that generates the language 

r r s t , , . 

tx y z : s + t = r>. 

f 5 t 

3. Is the language fx y z : s + t = r> deterministic 

context-free? Support your answer. 

r s t 

4. Is the language fx y z s r < s < t> context-free? Support 
your answer. 

5. Redesign the following grammar to obtain a context-free 

grammar that generates the same language and is suitable for an 
LL(1) parser. 

S -» xyzSz , 

S -* xyySy 
S -» xySw 
S -* X 

6. Describe the language accepted by the pushdown automaton 

represented below. What strings are accepted with the machine's 
stack empty? 
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7. Let G be a context-free grammar. Prove that there are 
positive integers m and n such that any string in L(G) of length 
more than m can be written as rstuv, where the length of stu is 

ft It 

less than n, either s or u is nonempty, and rs tu v is in L(G) 
for every k in M + . 

8. Show that the intersection of two context-free languages 
need not be context-free. 

9. Which of the following languages is not deterministic 
context-free? (w represents the string w written backward.) 

a. <ww: w € tx,y> and |w| < 5> 

jD |f 

b. Cww : w € fx,y> > 

|f 

c. fwyw: w € fx> > 

|f 

d. {ww: v/ € CkJ } 


J 


t 


/ 
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Chapter 3 


TURING MACHINES 
AND 

PHRASE-STRUCTURE LANGUAGES 


General Comments 


1. Try not to get bogged down in explaining composite Turing 
machines and the building blocks (such as S^» R^, etc.) 

introduced in this chapter. These are straightforward 

programming ideas that students can conquer mostly on their own. 
Instead, try to spend class time on the position of Turing 
machines in the hierarchy of language processing machines and as 
general computation devices. 

2. As in the case of pushdown automata, Turing machines can be 
consisdered as accepting a string in several different ways. It 
is important that students understand the equivalence of these 
acceptance criteria so that they will not become confused in 
later discussions. However, the point of the subsection "String 
Testing Procedures” in Section 3.3 is not to teach students how 
to convert machines that accept strings i^i one manner into 
machines that conform to another. Rather, the goal is merely to 
demonstrate that such conversions can be done. Keep in mind that 
the goal is to develop ideas regarding computability, not to 
teach skills in Turing machine programming. 

V x 

You may find a machine such as —>R-—->R »ALYL useful in 

ft 

class discussions. This machine accepts any string in (x,y> by 
halting; but if string acceptance is determined by halting with 
tape configuration AYAdd-••, then it accepts only the string yx. 

3. An important result in this chapter is that the generative 
power of grammars is exhausted at exactly the same level as the 
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language acceptance power of Turing machines. This is the first 
result presented in the text that supports Turing’s thesis. 
Chapter 4 continues this theme by showing how other hierarchies 
terminate at exactly the same position. Thus, Turing’s thesis is 
supported by much more than merely the fact that nobody has yet 
found a more powerful computational model. You might want to 
emphasize this point. 

4. Here is the grammar that is produced from the Turing machine 
of Figure 3.1? using the process described in the proof of 
Theorem 3.3. 


S -» Eh4Y43 

q AA -4 ApA 

qy4 -» yr4 

43 -4 443 

qxA -4 xp A 

qY4 -» Yr4 

Et.4 -» X 

qyA -4 ypA 

4s -t q4 

443 -» 43 

qYA -* Yp A 

tY -t s4 

43 -» X 

rA -4 qx 

h4Y -» 4tY 

41 -4 lA 

rA -4 qy 

hxY -» xtY 

xl -» lx 

qAA -4 ArA 

hyY 4 ytY 

yp -» iy 

qxA -4 xr4 

hYY -♦ YtY 

yp -* py 




Unfortunately, short examples of the grammar construction 
process described in the proof of Theorem 3.3 are not plentiful. 
After all, any Turing machine on which the construction is based 
must accept strings with its tape configured as 4Y444- ■ • . Thus, 
the machine must contain several transitions, each of which 
complicates the grammar. One example is given in the text as 
Exercise 3 at the end of Section 3.4. Here is another example. 
It is based on the following machine with alphabet L = tx> that 
accepts the language L and produces a manageable grammar. 
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The grammar produced is as follows: 


S -» Ch4Y43 
43 -» 443 
Cc4 -» X 
443 -» 43 
43 -» X 
4p -» 6 4 
kP -» px 

Using this grammar 
look like this. 

S => 
= > 
= > 
= > 
= > 
= > 
= > 
= > 
= > 
= > 
= > 
= > 
= > 
= > 
= > 


q44 -» 4p4 
qx4 -» xp4 
qY4 -4 Yp4 
r4 -» qx 
q44 -* 4r4 
qx4 -» xr4 
qY4 -4 Yr4 

a derivation for 

Ch4Y43 
Ch4Y443 
£ 4tY443 
t4s4443 
[q44443 
C4r4443 
C4qx443 
£4xr443 
E4xqx43 
C4xxp43 
£4xpx43 
C4pxx43 
c4xx43 
xx43 
xx 


4s -4 q4 
tY -4 s4 
h4Y -» 4tY 
hxY -4 xtY 
hYY -4 YtY 


the string xx would 


5. Some students may argue that the definition of (the 

language fw: w f L(M w >> ) provides an algorithm for detecting 
that a string belongs to (and hence Turing’s thesis must be 

wrong). More precisely, they may claim that all one must do when 
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given an input string w is compute the machine and then apply 

this machine to the input w. If the machine does not halt, then 
w € Lq, otherwise w f L^. 

The problem with this argument is that < in general) one 
cannot determine that M w will not halt. That is, a Turing 

machine does not indicate that it will never halt; it simply 
keeps running. 

This point is sometimes counterintuitive for students at 
this level. They have only written programs that, if correct, 
halt quickly. Their experience has taught them that if a program 
runs for more than a minute, it must be in an infinite loop. In 
turn, their intuition tells them that nontermination is 
detectable. You may wish to take class time to correct this 
misconception. If a program runs for a million years, it may or 
may not terminate. 

6. I included the composite diagram of a (three-tape) universal 
Turing machine (Figure 3.24) merely for the sake of completeness. 
I do not think it is worth class time. All I think students need 
is to see that such a machine is not beyond reason. You may want 
to point out that Figure 3.24 depicts a rather complex design. 
By developing other coding systems, extremely simple universal 
Turing machines are possible. (Perhaps some student would like 
to investigate this as an independent study project.) 

7. If you are running short on time, you can skip the proof 
that ail context-free languages are Turing-decidable. Instead, 
to obtain examples of Turing-decidable languages, you can show 
that all the regular languages are Turing-decidable. In fact, 
this makes a rather routine homework problem. (Merely start with 
a deterministic finite automaton that accepts the language in 
question and design a Turing machine that simulates this 
automaton’s actions except that, the Turing machine should answer 
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Y or N depending on whether or not the finite automaton reaches 
the end of its input in an accept state.) 

9. Some independent study projects relating to this chapter 
could be based on variations of Turing machines such as two 
dimensional tape machines or machines whose tapes extend 
indefinitely to the left as well as to the right. Characterizing 
context-sensitive languages in terms of Turing machines would be 
a good project for those students interested in the distinction 
between context-free and context-sensitive languages. 


Comments Regarding Chapter Review Problems 

Problems 10 and 11. The point of these problems is that the 
language generated by a grammar may be context-free, even though 
the grammar itself is not context-free. Some students seem to 
have trouble with this so I thought I should include these 
problems. Similar problems can be constructed in the context of 
regular languages. (The language generated by the grammar in 
Problem 10 is -Cx n+ *yx n ; n € W>, which is context-free. The 
language generated by the grammar in problem 11 consists of all 
strings over {x,y,z> having the same number of xs, ys, and zs, 
which is not context-free.) 

Problem 13. In short, the ability to suffer an abnormal 
termination is equivalent to the ability to reach the halt state. 

Problem 14. This would be equivalent to the ability to decide 
every Turing-acceptable language. But ' there are 
Turing-acceptable languages that are not Turing-decidable. 

Problem 15. The answer to part a is yes but the answer to part b 
is no. Every language is the union of (perhaps infinitely many) 
finite languages, each of which is Turing-acceptable. 


26 








Problem 16. As in problem 15 the answer to part a is yes but the 
answer to part b is no. Every language is the intersection of 
(perhaps infinitely many) languages formed by removing only a 

it 

finite number of strings from E . 

Problem 18. This problem is making the same point as problems 10 
and 11. 

Problem 19. One could simply add rewrite rules that had no 
effect on the original grammar. (This is certainly a simple 
problem, but it makes a point that many students may not have 
realized.) 

Problem 20. Students get the idea that examples of languages 
that are not Turing-acceptable are rare. (Just like many have 
never really thought about there being more irrational numbers 
than rational ones.) They should replace the picture below 


Languages that are 
not Turing-acceptable 



with this one. 



2 7 


ft 

Problem 25. Every language is a subset of E , which is 
Turing-acceptable. 

Problem 31. Perhaps the strings given in this problem are too 
long. The point is to make the students understand the algorithm 
rather then to force them to do a long, tedious calculation. 

Problem 32. See comment regarding problem 32. 

Problem 33. This is a preview of Chapter 5. 

Problem 34. Given any phrase-structure language there is a 
phrase-structure grammar that generates it. For each terminal x 
in this grammar, introduce a new nonterminal K and replace each 
occurrence of x in the grammar with X. Now introduce the rule 
X -* x for each of these new nonterminals. 

Problem 35. The purpose of this problem is to reinforce the term 
"recursively enumerable." 


Additional Problems 

I 

1. Show that any Turing-acceptable language is the language 
accepted by a Turing machine that never suffers an abnormal 
termination. 

2. Show that over any given alphabet there are infinitely many 
Turing-acceptable languages that are not Turing-decidable. 

i 

3. Show that every regular language is Turing-decidable. <Do 
not use the fact that every context-free language is 
Turing-decidable. ) 
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4. For each language L, is there a Turing machine that will 
reverse its input string before halting if that string is in L 
and not halt otherwise? Support your answer. 

5. Show that Turing’s thesis implies that the class of 
three-stack pushdown automata can have no more string processing 
power than the class of two-stack pushdown automata. 

6. Show that a multiple-tape Turing machine whose tapes extend 
indefinitely to the left as well as to the right has no more 
string processing power than a traditional (one-tape) Turing 
machine. 

7. Are there more Turing machines with alphabet fx,y> than 
there are with alphabet Cx>? Support your answer. 

8. Show that the class of "Turing machines" whose tapes contain 
only a finite number of cells would be able to accept only a 
subset of the regular languages. 
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Chapter 4 


COMPUTABILITY 


General Comments 

1. The purpose of this chapter is to present topics supporting 
Turing’s thesis, not to serve as a course on recursive function 
theory. Thus, I chose to introduce only those topics in 
recursive function theory that were required to develop the 
hierarchy of initial, primitive recursive, and partial recursive 
functions. Once again, my goal was to present an overall picture 
rather than get bogged down in the details of a particular topic. 

You will find, then, that several traditional topics (such 
as predicates, ^-recursive functions, and the s-m-n.theorem) are 
either not covered or only mentioned in the text. I encourage 
you to avoid branching out into these issues. Instead, stick to 
the global picture for this course and then specialize in later 
courses. 

2. In keeping with comment 1, I chose not to discuss bounded 
minimalization in the text. If this topiy is important to you, 
you may want to assign chapter review problem 25. It is the only 
reference to bounded minimalization in the text. 

3. When writing this text I put a major effort into finding the 
most efficient methods of presenting the material. However, I 
continue to find better ways of doing things--especially now that 
the final manuscript has been approved. One such improvement is 
in the proof that the constant functions k” are primitive 

recursive (Section 4.2, page 206). The text presents the double 
induction argument that I was taught many years ago. However, 
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all one actually needs is to observe that once K has been 

W! 

defined, K n can be defined for n > 1 as K® o r ' You may wish to 
’ m m 0 

use this simplified approach. 

4. As the years go by, the goto controversy becomes less and 
less of a topic of which our students are aware. After all, 
students who have learned to program in Pascal, Modula-2, or Ada 
may not even be aware that such a statement exists. However, I 
still think that studying the power of the goto statement is a 
worthwhile undertaking. This is why I like Exercise 1 at the end 
of Section 4.4. It describes a goto-less language with an 
expressive power equivalent to the primitive recursive functions. 
Moreover, this language is clearly a subset of the old standbys 
such as FORTRAN and COBOL. Thus, even when using these early 
languagess "most" problems can be solved without using goto 
statements. 

As an extension of Exercise 1, Section 4.4, you might want 
to ask your students to show that adding a goto statement expands 
the expressive power of the language in that exercise to that of 
any partial recursive function. <This is chapter review problem 
20 . ) 


5. You may want to point out to your students that the proof of 
Theorem 4.3 shows that any partial recursive function can be 
defined using only one minimalization step. (This is chapter 
review problem 27.) Indeed, if a function is partial recursive 
then it can be computed by a Turing machine. In turn, it can be 
defined by the process described in the proof of Theorem 4.3, 
which applies minimal izat ion only to find the number of steps 
required to reach the end of the machine’s computation. 

This observation allows one to define /vy[g(>T, y) = 01 as 
merely "the smallest y such that g(it,y) = 0" rather than “the 
smallest y such that g(iT f y) = 0 and g(x,z) is defined for all 

i 

z < y.” You may wish to use this as a way of launching your 
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better students into a research project. For example, they might 
investigate how the proofs in this chapter would change if this 
alternate definition of minimalization were used. 


6. Another independent study project for your better students 
would be to investigate the "Turing equivalence" of additional 
computational systems such as random access machines or the 
systems of Post and Markov. Such a study would provide further 
support for the Church-Turing thesis. 


7. Below is a diagram that I like to draw on the board as I 
review the material covered in chapters 1 through 4. The idea is 
for the students to see the relationships among the topics 
covered so far. The lecture that goes with the diagram is 
essentially my explanation of the diagram as I draw it. I leave 
this explanation to your own interpretation. 


f 

Algorithmically 

unsolvable 

problems 

4 

Church-Turing 
thesis 

A 

Algorithmically 

solvable 

problems 



V 


Languages & 
Grammars 


Functions Programming 

Languages 
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8. I recently taught a course in which some of the students 
confused the computability of a function with the existence of 
the function. For example, 1 found them saying that the function 
in chapter review problem 33 did not exist because there was not 
a way of deciding whether or not a general partial recursive 
function was defined for an arbitrary input. It is important 
that such misconceptions be revealed and corrected as soon as 
possible. In this case, a major point in the course is that 
there exist functions that are not computable! 

I think this example demonstrates how hard it is for some 
students to get away from the "computer science is programming” 
frame of mind that they have developed since high school or 
earlier. They have always been able to write a program to solve 
any problem given to them in class. It takes a while for them to 
realize that, with this course, the rules have changed. (I can 
remember a similar phenomenon when I was taking calculus. I 
never had to worry about whether or not the function given was 
integrable. If the problem required me to integrate it, I did 
it. It was in graduate school that I was finally forced to think 
about what I was doing. The idea that there are really functions 
that are not integrable had been foreign to me. Likewise, the 
idea that there are really functions that cannot be computed may 
be strange to your students.) 


Comments Regarding Chapter Review Problems 

Problem ?. The computation of Ackermann's function gets 
extremely complex as the inputs get larger than this example. I 
recommend that you avoid giving additional problems along this 
line. The time can be spent on other, more worthwhile, 
activities. 

i 


i 
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Problem 8. All I have in mind here is that Acfeermann’s function 
is an example of a total computable function that is not 
primitive recursive. 

Problem 11. The main step here is for students to realize that 
to show that g is partial recursive merely requires that they 
show it can be computed. 

Ue can compute g(y) by computing f(0), f(l>, f(2), etc. 
until the value x is reached for which f(x) = y. (If there is no 
such x, then g<y> is undefined and this process will never 
terminate.) Note that this approach relies on the fact that f 
(being primitive recursive) is total. Thus, if f(n) is not y, 
our process will discover this in a finite amount of time and 
move on to try the value n + 1. 

An interesting variation to this problem is to assume that f 
is one-to-one but only partial recursive. In this case, g(y) can 
be computed by performing the first step in computing f(0), then 
the first step in computing f(i), then the second step of f(0), 
the first of f(2), the second of f(l), the third of f(0), the 
first step of f(3), etc. 

Problem 12. There are uncountably many relations but only 
countably many computable relations. One relation that is not 
computable is the relation f(m,n); m is ^ natural number that 
represents a self-terminating Turing machine and n is a natural 
number that does not represent a self-terminating machine!. 

Problem 13. Pascal contains the bare-bones programming language 
presented in this chapter. However, when implemented on an 
actual machine, the size of integers is limited. 

i 

Problem 18. The point of this problem is to show that recursion 
is as powerful as iteration. 

Problem 20. See comment 4. 
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Problem 23. The function can be defined from the Turing machine 
by following the approach used in the proof of Theorem 4.3, 
except that minima 1ization is not required. 


Problem 27. See comment 5 


Problem 31. This is the closest that this text comes to 
discussing the s-m-n theorem. 


Problem 32. To compute h(x) merely alternate between the 
computation of f(x> and g(x). If either of these two 


computations terminates, halt with the output of one. 

Problem 33. The computation of h(x) would require the ability to 
decide whether or not f(x) is defined. But, the points at which 
many partial recursive functions are not defined cannot be 
determined algorithmically. Thus, the function h may not be 
partial recursive. 


Additional Problems 

1. Show that if f:IN -* IN is partial recursive, then the partial 
function g:IN -» IN defined by 



x if f(x) = y 
undefined otherwise 


g(y> 


is also partial recursive. 

2. Suppose that g:IN -» M is primitive recursive. Show that the 
function f s IN -* IN defined by 


f(x) = maximum of fg<y>: y < x> 


is primitive recursive 


35 




3. Suppose that g:IN -4 IN is partial recursive, 
f s IN -» IN defined by 


Is the function 


max tg<y>! g(y> is defined 
undefined otherwise 


partial recursive? Support your answer. 


Show that the function f:IN X IN -» IN defined by 


. J 1 if x < 

- (0 otherwi 


y 

se 


is primitive recursive, 


5. Suppose that g:IN -4 IN is primitive recursive, 
function f s IN -4 IN defined by 

x applications of g 
----^ 

f(x) = g(g< (g(x) )'•••)) 

is primitive recursive. (In other words, f(0) = 
f(2) = g(g(2)), etc.) 


6. Suppose that g:!N -4 IN is partial recursive, 
partial function f;IN -4 IN such that 


f <x> 


0 if x = 0 
x applications of g 

g(g(•■*(g(x)>•••)) when 0 < x and g( 
undefined otherwise 


is partial recursive. 


and y < x> 


Show that the 


, f(l> = g(1), 


Show that the 


t) is defined 
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7. Suppose g:(N -» IN is defined by g(x) = /uy[plus<x,y) = 03 and 
£ : IH -* IN is defined by f(x) = //yCmul t <0, g (x ) ) = 03. Describe the 
partial function f. 

8 . Show that any function that can be computed by a bare-bones 
program without using a while statement must be primitive 
recursive. 

9. Identify three hierarchies that we have used to support 
Turing’s thesis and explain the supporting factors. 

10. Give three equivalent definitions of a computable partial 
function. 

11. Suppose that L‘ is a Turing acceptable language whose 
complement is not Turing acceptable. Contrast the computability 
of the following functions. 


f <w> 



1 if w € L 1 
undefined otherwise 



1 if w 6 L’ 
0 otherwise 
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Chapter 5 


COMPLEXITY 


General Comments 


1. When introducing this chapter you could remind your students 
that many of the proofs in earlier chapters have involved 
constructions that would be long and tedious if actually carried 
out. Examples include the construction of a deterministic finite 
automaton that accepts the same language as a nondeterministic 
finite automaton and the construction of a context-free grammar 
that generates the same language that is accepted by a given 
pushdown automaton. Thus, these proofs show that such a 
construction is possible in a theoretical sense but not that such 
a construction is practical. This chapter, in contrast, is 
concerned with practicality of a computation. 

2. Another point to make when introducing this chapter is that 
the material to be presented is much newer than that in the other 
chapters. (Most of the material in the earlier chapters is 40 to 
50 years old.) Consequently, the classifications that will be 
presented in this chapter are not as clearly established as the 
hierarchies in the preceding chapters. In fact, this chapter 
introduces several unanswered questions that are subjects of 
current research. Thus, students should approach this chapter 
with an inquisitive mind rather than with the attitude of merely 
waiting for the answers to be revealed to them. 

i 

3. Most of the material in sections 1 through 3 will probably 
be familiar to (or at least easy for) students at this level. 
You can probably cover it in no more than two lectures--leaving 
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the majority of the time allotted to this chapter for sections 4 
and 5. 

4. I rarely present the entire proof of Cook’s theorem in 
class. Instead, I outline the proof, explaining the major ideas 
and providing enough details to convince the students that the 
proof is valid. Then, I encourage the students to read the proof 
on their own and return to ask questions. This approach allows 
more class time for discussing the meaning of Cook’s theorem—an 
activity that I consider more productive at this level than that 
of dwelling on the details of a long tedious proof. 

5. An activity suitable for an extra study project would be to 
investigate the details of Blum’s speedup theorem. However, only 
your better students (assuming undergraduates) will be able to do 
this on their own. An easier project would be to research the 
term HP - hard and its relationship to NP - complete . This project 
wi11 also have the side effect of encouraging students to 
consider the complexity classes above NP. Still another project 
could be based on a study of space complexity. 

6 . A subtlety in this chapter that many students do not defect 

is that the complexity of a computation, when measured as a 

function of the length of the input string, is dependent on the 

efficiency of the coding system being used. Suppose a problem 

deals with widgets, and the time required to solve an instance of 

the problem depends on the number of widgets involved--for 

instance, suppose that the presence of n widgets results in a 

2 

computation with time complexity n . Then, a coding system that 

represents the n widgets with a string of n symbols would make 

the computation appear to have a quadratic complexity, but a 

2 

coding system that required n symbols to represent the same n 
widgets would give the illusion of a linear complexity. (Also, 
see #4 in "Additional Problems" below.) 

I tried to avoid raising this issue in the text, but it is a 
point you may want to discuss with you class. 


39 




7. I hope that students understand from the early sections of 
this chapter that it is easy to establish an upper bound on the 
complexity of a problem (all one must do is find one solution) 
but it can be extremely hard to find a meaningful lower bound. 
This point is demonstrated by the detailed argument in Appendix 
D, where a lower bound for the string comparison problem is 
established in the context of one-tape Turing machines. I put 
this argument in an appendix because I did not feel that these 
details belonged in the flow of the text. (These arguments tend 
to require special, complex machinery that is rarely applicable 
to a broad range of problems. Thus, a lower bound for each 
problem must be established individually.) On the other hand, I 
think that seeing such an argument is probably the only way that 
a student will truly realize the difficulty of establishing such 
lower bounds. 

1 encourage you to discuss this lack of symmetry in 
establishing bounds with your students, perhaps in the context of 
explaining that problems are more normally classified in terms of 
0(f) rather than 8( f). Moreover, I suggest giving Appendix D as 
a reading assignment. You might even follow up with a few 
questions about crossing sequences. But, I would not recommend 
committing class time to the details of Appendix D. There are 
too many general ideas in this chapter that are more worthy of 
class discussion time. 


Comments Regarding Chapter Review Problems 

Problems 1 through 5. These are straightforward problems dealing 
with the material in the early sections. 
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Problem 6. For any n > 2d we have 

nf = n(n - l)(n - 2)---(n - d + l)--2 

> n(n - 1)(n - 2)-••<n - d + l)2 d 

= 2n2(n - l)2<n - 2)---2<n - d + 1) 

> n d 


Problem 7. log a n = Hog^ bHlog^ n> 


Problem 8. The function log n is in O(n) but not in ®(n ) for 
any d € IN + . 

Problem 9. All context-free languages are in P. Consequently, 
if your students can show that Lg^ is context-free, they will 

have shown that P equals NP. 

Problem 12. 0 is accepted in polynomial time by the Turing 

£ 

machine -»L. E is accepted in polynomial time by the Turing 

machine -*R. 


Problem 13. There are no strings in 0 to be the image of those 
strings in E - fx>. 

Problem 14. This is another crossing sequence problem. 


Problem 21. The sum of two polynomials is a polynomial. 

Problem 22. Some students will propose "dropping the z," and in 
fact this will produce a palindrome from strings of the form 

jj 

wzw . But, this technique does not provide a function whose 

ft 

domain is all of fx,y,z> . (What should the reduction do to the 
string xxx?) Moreover, "dropping the first z if there is one" 
does not work either since it produces palindromes from the 

JJ 

strings xzyyx and xxx, which are not in the form wzw . 

/ 
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Problem 23. The function f(w) = xx, where w 6 Cx,y> , does the 
job. 

it 

Problem 24. f(w) = wx, where w € tx,y} . 

Problem 27. This is just a way of asking the student to remember 
that the class P is the same regardless of how many tapes are 
used by the Turing machine. 

Problem 30. Suppose L 0 is HP-complete and its complement, L^, is 

in HP. Then, for any language L in co-HP, there is a polynomial 
reduction from the complement of L to Lq, which in turn provides 

a polynomial reduction from L to L^. By first applying this 

Q 

reduction, any nondeterministic Turing machine that accepts in 
polynomial time can be used to accept L in polynomial time. 

Problem 31. Design a deterministic Turing machine that traces 
all the paths in the nondeterministic machine far enough so that 
acceptance by the nondeterministic machine would be detected. 

Problem 32. Show that the process of deciding a context-free 
language as described in Section 3.5 can be accomplished in 
polynomial time. j 


Additional Problems 

1. Is there a polynomial-time algorithm for constructing a 

/ 

deterministic finite automaton that accepts the same language as 
that accepted by a given nondeterministic finite automaton? 

2. What is the time complexity of determining whether or not a 
given finite automaton is deterministic? 
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3. Classify the time complexity of determining whether or not a 
given string is a valid regular expression. 

4. Describe a coding system with which the time complexity of 
the traveling salesman decision problem is a linear function of 
the length of the input string. 

5. Suppose there is a reduction from the language L, to the 

language that can be computed by a Turing machine in no more 

than 2 n steps, where n is the length of the input string. 
Furthermore, suppose can be accepted by a Turing machine in no 

more than 2™ steps, where m is the length of the input string. 
Uhat can be said about the time complexity of accepting the 
language L^? 

6. Suppose £ = fx>. Show that there is not a polynomial 

if 

reduction from the language fx> to the language L . 

7. Show that 0(2 n ) = 0(2 n+1 >. 

8. Design a two-tape Turing machine with alphabet L that 

R ft 

accepts the language tww : w € £ > with time complexity in 0(n). 

9. Show that every regular language can be accepted by a Turing 
machine with linear space complexity. 

10. Suppose that and are languages in P. Show that 

Lj H Lj is also in P. 

0 n 

11. Suppose f(n) = n , g(n> = 2 , and h(n) = nf. Identify the 
relationships between 0(f), 0(g), and 0<h). Then, do the same 
for 0(f), 0(g), and 0(h). 
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TRANSPARENCY MASTERS 






Figure 0.2 


procedure sort (list); 
begin 

if (list not empty) then 
begin 

sort (portion of list following first entry); 
move first entry to its proper place within 
the following portion of the list 
end 

end; 


© 1989 Benjamin/Cummings Publishing Company, Inc. 


Figure 1.2 


State : = 1; 

Read the next symbol from input; 

While not end-of-string do 

Case State of 

1: If the current symbol is a letter then State := 3, 
else if the current symbol is a digit then State : = 2, 
else exit to error routine; 

2: Exit to error routine; 

3: If the current symbol is a letter then State : = 3, 
else if the current symbol is a digit then State ; = 3, 
else exit to error routine; 

Read the next symbol from the input; 

End while; I 

If State not 3 then exit to error routine; 


i 
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Figure 1.3 



letter 

digit 

EOS 

1 

3 

2 

error 

2 

error 

error 

error 

3 

3 

3 

accept 
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Figure 1.4 


State := 1; 

Repeat 

Read the next symbol from the input stream; 
Case symbol of 
letter: Input : = “letter"; 
digit: Input : = “digit”; 
end of string marker: Input : = “EOS”; 
none of the above: exit to error routine; 
State := Table[State, Input]; 

If State = “error” then exit to error routine; 
until State = “accept” 1 
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Figure 1.5 
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Figure 1.6 



digit 

• 

E 

+ 

— 

EOS 

1 

2 

error 

error 

error 

error 

error 

2 

2 

3 

5 

error 

error 

error 

3 

4 

error 

error 

error 

error 

error 
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4 

error 

5 

error 

error 

accept 
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7 

error 

error 
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6 

error 

6 

7 

error 

error 

error 

error 

error 

7 

7 

error 

error 

error 

error 

accept 














Figure 1.7 


State : = 1; 

Repeat 

Read the next symbol from the input stream; 
Case symbol of 
0 through 9: input: = “digit”; 

End of string marker: input : = “EOS”; 

• ,E, +, -: Input : = symbol; 

None of the above: exit to error routine; 
State := Table[State, Input]; 

If State = “error” then exit to error routine; 
until State = “accept” 


/ 
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Figure 1.10 
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Figure 1.11 
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Figure 1.19 


y 
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Figure 1.20 


< sentence > — < subject > < predicate > < period > 

< subject > — < noun > 

< noun >—John 

< noun > — Mary 

< predicate > — < intransitive verb> 

< predicate > — < transitive verb> < object > 

< intransitive verb >— skates 

< transitive verb > —hit 

< transitive verb >-* likes 

< object > — <noun> 

< period > —. 


Figure 1.21 


< sentence > 

iy 

< subject > < predicate > < period > 

< noun > < predicate > < period > 

'U' 

Mary < predicate > < period > 

V 

Mary < transitive verb> < object > < period > 

O 

Mary likes < object > < period > 

l 

Mary likes < noun > < period > 

V 

Mary likes John < period > 

Mary likes John. 
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Figure 2.8 
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Figure 2.9 
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Figure 2.12 


From rule 1 
S— <f, X, h> 


From rule 2 

<f, X, f>-X 
<9, X, g>-X 
<h, X, h>~\ 


From rule 3, transition (g, c, c; h, X) 

<g, c, f>-c<h, X, f> 

<9, c, g>—c<h, X, g> 

<g, c, h>~c<h, X, h> 


From rule 4, transition (f, c, X; g, c) 

<f, X, f> - c <g, c, <f, X, f> 
<f, X, f> - c <g, c, g> <g, X, f> 
<f, X, f> - c <g, c, /?> </7, X, f> 
<f, X, g> - c <g, c, f> <f, X, g> 
<f, X, g> — c <g, c, g> <g, X, g> 
<f, X, g> - c <g, c, h > <Aj, X, g> 
<f, X, /?> -* c <g, c, <f, X, h > 
<f, X, /i> - c <g, c, g> <g, X, 

<f, X, h> — c <g, c, /)> </>, X, 


<f, c, f> — c <g, c, f> <f, c, f> 
<f, c, f> —» c <g, c, g> <g, c, f> 

<f, c, f> - c <g, c, A)> </», c, f> 

<f, c, g> —» c <g, c, f> <f, c, g> 
<f, c, g> -» c <g, c, g> <g, c, g> 

<f, c, g> - c <g, c, /j> <h, c, g> 

<f, c, h> - c <g, c, <f, c, 

<f, c, h> - c <g, c, g> <g, c, /?> 

<f, c, Aj> - c <g, c, </?, c, h> 


From rule 4, transition (g, b, \;g, X) 


<g, x. 

f> 

— 

b 

<9, X, 


<f, X, f> 

<9, 

c, 

f> 

— 

b 

<9, X, 

f> 

<f, c, f> 

<9, 

X, 

f> 

— 

b 

<9, X, 

g> 

<9, x, 

<9, 

c, 

f> 


b 

<g, X, 

9> 

<9, c, f> 

<9, 

X, 

f> 

— 

b 

<9, X, 

h > 

<ft, X, f> 

<9, 

c, 

f> 

— 

b 

<g, X, 

h> 

<h, c, f> 

<9, 

X, 

9> 

— 

b 

<g, x. 

f> 

<f, X, g> 

<9, 

c, 

9> 

— 

b 

<g, x, 

f> 

<f, c,g> 

<9, 

X, 

9> 

— 

b 

<9, X, 

g> 

<g, x, g> 

<9, 

c, 

9> 

— 

b 

<g, x, 

g> <9, C, g> 

<9, 

X, 

9> 

—* 

b 

<9, X, 


<h, X, g> 

<9, 

c, 

9> 

— 

b 

<g, x, 

h> <h, c, g> 

<9, 

X, 

h> 

— 

b 

<g, x, 

f> 

<f, X, /)> 

<9, 

c, 

h> 

— 

b 

<g, x, 

f> 

<f, c, h> 

<9, 

X, 

h> 

— 

b 

<9, X, 

9> 

<g. x, h> 

<9, 

c, 

h > 

— 

b 

< 9 , x, 

9> 

<g,ic, h> 

<9, \ 

h> 

— 

b 

<9, X, 

h> 

<h, h> 

<9, 

c, 

h> 

— 

b 

<g, x, 

h> 

<h, c, h> 
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Figure 2.25 
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Figure 2.26 


State : = i; 
push (#); 

State : = p; 
push (S); 

State : = q; 

while top-of-stack =£ # do 
case top-of-stack do 

S: either pop (S) and push (xSy) 
or pop (S); 

x: pop (x), read an x from the input; 
y: pop (y), read a y from the input; 
end case 
end while; 
pop (#); 

State : = f 
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Figure 2.27 


State : = t; 
push (#); 

State : = p; 
push (S); 

State : = q; 
read (Symbol); 
while top-of-stack =£ # do 
case top-of-stack of 

S: if Symbol =£ x then pop (S) 
else pop (S), push (xSy); 
x: if Symbol not x then exit to error routine 
else pop (x), read (Symbol); 
y; if Symbol not y then exit to error routine 
else pop (y), read (Symbol); 
end case 
end while; 
pop (#) 

if Symbol not end-of-string marker then exit to error routine; 
State : = f 


Figure 2.32 
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Figure 2.34 
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shift 2 
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shift 3 


shift 7 
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shift 3 


shift 7 
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shift 9 
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shift 9 
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Figure 2.35 


Token := 1; 
push (Token); 
read (Symbol); 

TableEntry : = Table [Token,Symbol]; 
while TableEntry is not “accept” do 
if TableEntry is a shift 
then begin 

push (Symbol); Token : = TableEntry.State; 

push (Token); read (Symbol) 

end 

else if TableEntry is a reduction 
then begin 

pop (right side of TableEntry.RewriteRule); 

Token := top-of-stack; (*This is not a pop*) 
push (left side of TableEntry.RewriteRule); 

Token : = Table [Token, left side of TableEntry.RewriteRule]; 

push (Token) 

end 

else if TableEntry is blank then exit to error routine; 

TableEntry : = Table [Token,Symbol]; 
end while; 

if Symbol not EOS then exit to error routine; 
empty stack 


Figure 2.37 
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Figure 2.38 
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Figure 3.14 
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Figure 3.15 
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Figure 3.17 




[AxIxyA] 

[AxxlyA] 

[AxxypA] 

[AxxqyA] 

[Axx/A] 

[AxqxA] 

[Ax/A] 

[ApxA] 

[A/A] 

[QA] 

[ASA] 

[AtYA] 

[hAYA] 


( 


I 
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Figure 3.18 


S => [hAYA] 
[hAYAA] 
=> [/7AVAAA] 
[Af YAAA ] 
=> [AsAAAA] 
[qAAAAA] 
=£> [ArAAAA] 
=> [AqxAAA] 
[AxrAAA] 
[AxqxAA] 
z$> [AxxrAA] 
[AxxqyA] 
[AxxypA] 
[Axx/yA] 
r£> [Ax/xyA] 
i£> [A/xxyA] 
=£> [iAxxyA] 
=£> xxyA] 

=> xxy 
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Figure 3.23 


This cannot be the beginning 
of another transition since that 
interpretation would produce 
an invalid state code. 



1st transition 2nd transition symbols 


l 


/ 
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Figure 3.26 


S-MN 

M-MP 

N—PN 

M-x 

N~y 

P-x . 
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xxxx 
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xxyyi 


xyyy 

xyyx 

xyyy 1 
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Figure 3.31 
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phrase-structure 

languages 


Turing-decidable 

languages 


context-free 

languages 


deterministic context- 
free languages 


regular languages 
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Figure 4.9 



all functions 
mapping 1N/» intolN* 


partial recursive 
functions 


/t-recursive functions 


primitive recursive 
functions 


initial functions 
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Figure 4.11 


A machine to compute a : 



A machine to compute f: 


—► ROL 


A machine to compute xj: 


i times i - j times 

j - 1 times 

i — —I i— “I i 

_U „ 'I * I . , _ J 

A 

i a i i r -| 1 ‘ ' 

L_ J L. 

s l l —. 
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Figure 4.12 


step 2a 


step 2b 


step 2c 


A x A y+1 A ... 

t 


A x A y A ... 

t 

A x A y A x A y-1 A 


A x A y A x A y-1 A 


A x A y A x A y-1 A 


A x A y A x A y-1 A 


A X A y A x A y-1 A 


A x A y A x A y-1 A 


A X A y A x A y-1 A 


..AxAOAxA... 

t 

..AxAOAxA... 

t 

. . A X A 0 A g(x) A . 
. . A X A 0 A /(X,0) A 
. . A X A 0 A /(X, 0) A 

.. A h(x,0,f(x,0)) A . . 

t 

. . A /(x,1) A . . . 


A X A y A /7(x,y,/(x,y-1)) A ... 
A X A y A /J(x,y,/(x,y-1)) A ... 
A x A y A /(x,y) A ... 

t 

A h(x,y,f(x,y)) A ... 

A f(x,y+ 1) A ... 

t 
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Figure 4.16 


G 

aux—x^ +1 ; 
clear x k+1 ; 
while aux =£ 0 do; 

X k+2*~ Z V 

X k+2~~ Z 2' 


Xk + m+l *~~ Z m » 

H 

incr x* +1 ; 
deer aux; 


end; 
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Figure 4.17 


clear x n+1 ; 

G 

while z 1 ^ 0 do; 
incr x n+1 ; 

G 

end; 

Zi-* n+ i 
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Figure 5.3 
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Figure 5.5 


program InsertSort (inout: List; in: ListLength) 
var 

PivotPosition, I: integer; 

Pivot: ListEntryType; 

begin 

if (ListLength a: 2) then 
begin 

PivotPosition := 2; 
repeat 

Pivot : =* List[PivotPosition]; 

I := PivotPosition; 

while (I > 1 and List[l - 1] > Pivot) do 
begin 

List[l] := ListJI - 1]; 

I := I - 1 
end; 

List[l] := Pivot; j 

PivotPosition : = PivotPosition + 1 
until (PivotPosition > ListLength) 
end. 


i 
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Figure 5.6 
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Figure 5.13 
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Figure 5.15 


truth assignment 


v 3 = true 
v 2 = false 
v, = false 


ir u 

n n p 




k i 

i k \ 

match 

match 

r ' 

r u 


coded clauses 


match 

(pOO/OnO) (OpO/OOp) (nOO/OOn/OpO) 
clause 1 clause 2 clause 3 


© 1989 Benjamin/Cummings Publishing Company, Inc. 












Figure 5.16 


Starting 

configuration 

Tape 1: A(p00/0n0)(0p0/00p)(n00/00n/0p0)AA 

Tape 2: AAA ... 

4 

Truth assignment 
has been written 
on tape 2 

Tape 1: A(p00/0n0)(0p0/00p)(n00/00n/0p0)AA 

Tape 2: AnnpAA ... 

First clause is 
found to be 
satisfied 

Tape 1: A(pOO/OnO)(OpO/OOp)(nOO/OOn/OpO)AA 

Tape 2: AnnpAA ... 

Second clause is 
found to be 
satisfied 

Tape 1: A(p00/0n0)(0p0/00p)(n00/00n/0p0)AA 

Tape 2: AnnpAA... 

4 

Third clause is 
found to be 
satisfied 

Tape 1: A(pOO/OnO)(OpO/OOp)(nOO/OOn/OpO)AA 

! 4 

Tape 2: AnnpAA ... 

Machine halts 
having found all 
clauses to be 
satisfied 

Tape 1: A(p00/0n0)(0p0/00p)(n00/00n/0p0)AA 

Tape 2: AnnpAA ... 


/ 
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Range of 


Variable 

subscripts 

Hd, , 

0 < / < p(n) 

1 < / < p(n) +1 

Shi 

0 < / < p(n) 

1 < j < r 

Conti t „ 

0 < / < p(/7) 
i < y < p(n) + 1 
1 < k < m 


Corresponding 

statement 


Number of variables 
of this type 


M’s tape head is 
over cell j at 
time /. 

M is in state g, 
at time /. 

Cell j contains 
symbol x k at 
time /. 


[p(n)+1] 2 

r[P(n) + 1 ] 
m lP{n) +1 ] 2 



Figure 5.18 


Constraint 

M’s tape head must 
be over at least one 
cell at any time. 


M’s tape head can 
be over only one cell 
at any time. 


M must be in at least 
one state at any time. 


M can be in only one 
state at any time. 


Each tape cell must 
contain at least one 
symbol at any time. 


Each tape cell can 
contain at most one 
symbol at any time. 


Number of 

Clauses_such clauses 

Hdj , or Hdf 2 or... or Hd, t m+ , p(n) +1 

for each / in {0, 1, p(n )) 


Hdj j or Hd\ k (p(n) +1) 2 p(n) 

for each / in (0, 1, ..., p(n )) 2 

and each j and k in 

11.2.PM+11 

with j < k 


St it ! or Stj 2 or... or Sf,, p(n) +1 

for each / in (0, 1, ..., p(n)) 


Stj j Or Stj k 

for each / in {0, 1, , p(n)\ and 

each j and k in (1, 2. r\ with 

/ < k 


(p(n)+1) 


r(r- D 

2 


Cont it /; i or Cont it jt 2 or ... or Cont it jt m (p{n) +1) 2 

for each / in (0, 1, , p(n)\ j 

and each / in (1, 2. p(n) +1) 


Cont it j' k or Cont^ u , 

for each / in (0, 1, ..., p(n)) t 
each j in {1, 2, ..., p(r7) + 1) f and 
each k and / in (1, 2, ..., m ) with 
k < I 


(m+vslepl 
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Figure 5.19 
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Figure A.1 
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